BMJ 



'Department of Gynecology, Juliane 
Marie Centre for Children, Women 
and Reproduction, Copenhagen 
University Hospital Rigshospitalet, 
Blegdamsvej 9, DK-2100 OE, 
Copenhagen, Denmark 

department of Obstetrics, Juliane 
Marie Centre, Copenhagen 
3 Division of General Surgery, St 
Michael's Hospital, Toronto, ON, 
Canada 

department of Gynecology, 
Juliane Marie Centre, Copenhagen 

department of Vascular Surgery, 
Abdominal Centre, Copenhagen 
University Hospital Rigshospitalet, 
Copenhagen 

Copenhagen University Hospital 
Rigshospitalet, Copenhagen, 
Denmark 

Correspondence to: C R Larsen 
crl@dadlner.dk 

Cite this as: BMJ 2009;338:b1802 

doi:10.1136/bmj.b1802 



RESEARCH 



Effect of virtual reality training on laparoscopic surgery: 
randomised controlled trial 

Christian R Larsen, clinical research fellow, 1 Jette L Soerensen, assistant professor and consultant, 2 Teodor P 
Grantcharov, assistant professor and consultant, 3 Torur Dalsgaard, consultant, 4 Lars Schouenborg, 
consultant, 4 Christian Ottosen, consultant, 4 Torben V Schroeder, professor and consultant, 5 Bent S Ottesen, 
managing director and professor at the Juliane Marie Centre 6 



ABSTRACT 

Objective To assess the effect of virtual reality training on 

an actual laparoscopic operation. 

Design Prospective randomised controlled and blinded 

trial. 

SettingSeven gynaecological departments in the Zeeland 
region of Denmark. 

Participants 24 first and second year registrars 
specialising in gynaecology and obstetrics. 
Interventions Proficiency based virtual reality simulator 
training in laparoscopic salpingectomy and standard 
clinical education (controls). 

Main outcome measure The main outcome measure was 
technical performance assessed by two independent 
observers blinded to trainee and training status using a 
previously validated general and task specific rating 
scale. The secondary outcome measure was operation 
time in minutes. 

Results The simulator trained group (n=ll) reached a 
median total score of 33 points (interquartile range 32-36 
points), equivalent to the experience gained after 20-50 
laparoscopic procedures, whereas the control group 
(n=10) reached a median total score of 23 (22-27) points, 
equivalent to the experience gained from fewer than five 
procedures (P<0.001). The median total operation time in 
the simulatortrained group was 12 minutes (interquartile 
range 10-14 minutes) and in the control group was 24 
(20-29) minutes (P<0.001). The observers' inter-rater 
agreement was 0.79. 

Conclusion Skills in laparoscopic surgery can be 
increased in a clinically relevant manner using proficiency 
based virtual reality simulator training. The performance 
level of novices was increased to that of intermediately 
experienced laparoscopists and operation time was 
halved. Simulator training should be considered before 
trainees carry out laparoscopic procedures. 
Trial registration ClinicalTrials.gov NCT00311792. 

INTRODUCTION 

Laparoscopy has become the standard approach for 
many conditions in most surgical specialties. 1 3 This 
development has been driven by the desire for less sur- 
gical trauma, faster postoperative recovery, shorter 



hospital stay, and better cosmetic results, and a sales 
drive by the medical industry. 4 It is evident, however, 
that laparoscopy is associated with a longer operation 
time and a higher rate of surgical complications during 
the learning curve of the surgeons. This has been ver- 
ified in many different specalties, including general 56 
urological, 78 paediatric, 9 and gynaecological 
surgery. 10 The possibility of overcoming these pro- 
blems during the learning curve by appropriate train- 
ing and ensuring that surgeons perform a sufficient 
number of procedures has also been documented. 11 
The technical skills needed for laparoscopic surgery 
are fundamentally different from those for traditional 
open surgery, leading to a prolonged learning curve. 
The primary obstacles in learning laparoscopy are psy- 
chomotor and perceptual. The unique nature of 
laparoscopic surgery combined with an increasing 
focus on patients' safety and rights, the present 
decrease in working hours, and concern over costs of 
operating theatre time are factors that challenge the 
traditional surgical approach and contribute to a grow- 
ing need for novel methods in the training of laparo- 
scopic surgeons. 12 Although virtual reality simulation 
has the potential to offer important advantages in the 
area of training for new skills and procedures, evidence 
on the transfer of skills from the simulated environ- 
ment to the operating theatre is still limited. 13 14 

We investigated the impact of training using a virtual 
reality simulator on the quality of skills acquired for a 
key gynaecological procedure. The investigation was 
carried out as a prospective, randomised, observer 
blinded, controlled trial, according to the guidelines 
of the consolidated standards of reporting trials 
(www.consort-statement.org). 

METHODS 

From September 2006 to June 2007, 32 trainees in 
gynaecological specialty training years 1 and 2 (post- 
graduate years 3-8; see box), with no experience of 
advanced laparoscopy (defined as all laparoscopic pro- 
cedures involving coordination of more than one 
instrument), were included in the study. Of a total 
cohort of 42 (38 women and four men) trainees in the 
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region, eight were not eligible, as they were too experi- 
enced and four came from the two departments that 
did not participate in the trial. Of the remaining 30 
eligible trainees, the first 24 who volunteered were 
enrolled. They came from seven of nine gynaecology 
departments in the Zeeland region of Denmark (popu- 
lation 2.3 million): Gentofte hospital (five trainees), 
Herlev hospital (n=4), Roskilde hospital (n=4), Hiller- 
oed hospital (n=l), Holbaek hospital (n=l), Hvidovre 
hospital (n=2), and Rigshospitalet hospital (n=7). 

Randomisation and blinding 

To ensure that the trainees' baseline characteristics 
were similar within and between each group, we 
chose a stratified randomisation based on previous 
experience of simple laparoscopy (defined as laparo- 
scopic procedures performed using a single instru- 
ment, such as diagnostic laparoscopic sterilisation 
(clips) or diagnostic laparoscopy). The Clinical Trial 
Unit at Copenhagen University independently rando- 
mised the trainees by computer to intervention or con- 
trol groups. The randomisation procedure was 
concealed and achieved by using the trainees' unique 
personal identification number (central personal regis- 
ter number). 

Trainees in the intervention group were given an 
oral introduction to the simulator and the rating scale 
used for outcome measure. Any operations done dur- 
ing the study were recorded. Owing to the nature of the 
trial it was not possible to blind the trainees to their 
allocated group, but all involved departments, super- 
visors, and staff in the operating theatres were blinded 
to the trainee's group, and the assessors of outcome 
were blinded to both the trainee and their allocated 
group. The primary investigator saw the data only 
after completion of all assessments and once data had 
been loaded in a database. 

The control group was to continue standard clinical 
education. During the study no trainee in either group 
was allowed to perform advanced laparoscopic sur- 
gery, only simple laparoscopy or to assist senior collea- 
gues. To check that randomisation had been 
successful, the control group were trained in the simu- 
lator after the trial. Their baseline performances were 
indistinguishable from those of the intervention group. 

Equipment 

The virtual reality laparoscopy simulator program 
(LapSim Gyn v 3.0.1; Surgical Science, Gothenburg, 
Sweden) was run on an IBM T42 computer in a dock- 
ing station (Pentium M 1.8 GHz/5 12 MB RAM; IBM, 

Duration of specialist training in Denmark 

Preregistration house year— one year 
Introduction before specialist training — one year 

Optional additional training to quality for further specialisation (possibility for writing PhD 
thesis) — 1-3 years 

Specialist training — four years 



Armonk, NY, USA) using an interface with a dia- 
thermy pedal (Virtual Laparoscopic Interface; Immer- 
sion, San Jose, CA, USA). The operations took place in 
the operating theatres of the participating departments 
and were recorded on DVD using a camera attached to 
the laparoscope for later blinded evaluation. During 
the operation one of the authors (CRL or designated 
TD) observed the procedure to record the handling of 
the surgical instruments, any involvement of the super- 
visor, whether the standard procedure for the opera- 
tion was followed, and whether the recording was 
done correctly, finalised, and assessed. 

Simulator training 

The intervention group undertook a specific training 
programme in the simulator. The programme com- 
prised two parts: firstly, training in the two basic skills 
of "lifting and grasping" and "cutting" during which the 
trainees were introduced to the simulator environment 
and the different instruments; secondly, one procedure 
specific task in which the trainee had to carry out a 
complete right sided salpingectomy while preserving 
the ovary. The training in basic skills was done once 
in each training cycle of 45-60 minutes and the salpin- 
gectomy repeated continually during the remainder of 
the cycle. The simulator provided the trainees with 
instant feedback on time, path length and angular 
path of the instruments' movements, bleeding, cutting 
of uncoagulated arteries, and use of diathermy on non- 
target tissue. The training sessions were repeated until 
the expert criterion level was reached in two consecu- 
tive and independent simulations. The proficiency cri- 
teria were established by experts in previous studies of 
construct validity and learning curves. 1 5 16 The require- 
ments and settings of the simulator are available at 
www.skopisimulator.rh.dk. 

Surgical procedure 

The trainees performed their first salpingectomy at 
their local gynaecological department and were super- 
vised by a senior colleague who was told about the 
purpose of the trial. To make comparison of perfor- 
mance easier, the trainees all carried out procedures 
on the right side. The patients were admitted for elec- 
tive salpingectomy before treatment for infertility or 
for prophylactic removal of fallopian tubes and ovaries 
owing to a positive test result for breast cancer gene 1 
(BRCA1). The trainees were not allowed to operate on 
patients who had undergone previous open or laparo- 
scopic surgery below umbilicus, had possible abdom- 
inal malignant disease, had an American Surgical 
Association score >3 (patients with severe systemic dis- 
ease), had a body mass index less than 18 or more than 
27, had haemophilia, or had other factors of potential 
influence on the surgical procedure. The operations 
followed a modified standard procedure on the basis 
of expert consensus. 17 18 The supervisors were allowed 
to give oral instructions only, and one researcher was 
present to observe the procedure and to record who 
handled the instruments. 
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Inclusion of trainees and patients (n=24) 



Concealed randomisation (n=24) 



_L 



Simulated training until two 
consecutive scores at expert level 
Maximum 60 days (n=13) 



Conventional clinical education 
Assistance in operating theatre 
Maximum 60 days (n=ll) 



Laparoscopic salpingectomy 
Unedited DVD recording (n=13) 



Laparoscopic salpingectomy 
Unedited DVD recording (n=ll) 



Drop outs (n=2): 
Failed to fulfil training 
programme (n=l) 
Patient ineligible (n=l) 



Assessment of video recordings 
by two independent observers 
blinded to trainee and training 
status using objective structured 
assessment of laparoscopic 
salpingectomy (n=ll) 



Drop outs (n=l): 
Video recording failed (n=l) 



Assessment of video recordings 
by two independent observers 
blinded to trainee and training 
status using objective structured 
assessment of laparoscopic 
salpingectomy (n=10) 



Simulator training control group (postoperative or voluntary) (n=9) 



Flow of trainees through trial 



Outcome measures 

The primary outcome measure was technical perfor- 
mance, measured as total score (10-50 points) using 
the objective structured assessment of laparoscopic sal- 
pingectomy, which comprises a five item general rat- 
ing scale and five item task specific rating scale. 19 Two 
independent observers blinded to trainee and allo- 
cated group assessed the recorded operations. The sec- 
ondary outcome measure was operating time in 
minutes. The reliability of the assessment was deter- 
mined by calculating the inter-rater agreement (num- 
ber of agreements for each of the assessed items 
divided by total number of assessed items) and the y 
coefficient. We present outcomes as medians and inter- 
quartile ranges. 

Power calculation 

The power calculation was based on a previous valida- 
tion study on the procedure specific scale of the objective 
structured assessment of laparoscopic salpingectomy. 19 
This study showed a difference of six points between 
novice laparoscopists (0-5 procedures) and inter- 
mediately experienced laparoscopists (30-50 proce- 
dures). An improvement of skills to the level of 30 or 
more points was considered acceptable. On the basis of 
these findings we chose the minimal relevant difference 
to be six points. We determined that with an a of 0.05 



Table 1 1 Baseline characteristics of gynaecology trainees randomised to virtual reality 
simulator training in laparoscopic salpingectomy or to standard clinical education (controls). 
Values are numbers of trainees unless stated otherwise 


Characteristics Simulator trained group (n=13) 


Control group (n=ll) 


Men 1 


1 


Women 12 


10 


Mean (range) age (years) 33.3 (30-42) 


32.4 (26-38) 


Experience of simple laparoscopy 6 


5 


No experience of simple laparoscopy 7 


6 



(two sided) and a power of 80% ((5=0.2 giving Za=1.96 
and Zp=0.84, largest SD=4.40) we required 18 or more 
trainees. To compensate for possible drop outs, we 
added an additional third to the 18, totalling 24 trainees. 

Statistical analysis 

We present cumulated scores as medians (average 
score of two observers), compared using non-para- 
metrical analysis (Mann- Whitney U test). We consid- 
ered a two tailed P value of 0.05 or less to be statistically 
significant and an inter-rater agreement and y coeffi- 
cient of 0.8 or more for each to be acceptable. Analysis 
was done using SPSS 13.0 for Windows. Graphics were 
made using Graph pad Prism (Graph Pad, San Diego, 
CA, USA). 

RESULTS 

Eight of the total cohort of 42 trainees (38 women, four 
men) were ineligible for the study as they were too 
experienced and four came from the two departments 
not participating in the trial. The remaining 30 trainees 
agreed to participate. The first 24 were enrolled; 22 
(90%) were women, representing the current sex distri- 
bution among trainees in obstetrics and gynaecology 
in Denmark (figure). The average age of the trainees 
was 32.8 years (range 26-42 years), and 23 were right 
handed. Eleven trainees were randomised to virtual 
reality training in laparoscopic salpingectomy and 10 
were randomised to traditional clinical education. 
Table 1 shows the baseline characteristics of the trai- 
nees. Two trainees were subsequently excluded from 
the simulator trained group because one failed to com- 
plete the training programme and the other was 
involved in an operation that was cancelled because 
of anatomical abnormality and suspected malignant 
disease in the patient. One trainee was excluded from 
the control group because of a technical fault in the 
DVD recorder used to record the operation. 
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Table 2 j Impact of virtual reality simulator training on surgical performance and operation 


time. Values are medians (ranges; interquartile ranges) unless stated otherwise 






Simulator trained group 


Control group 




Outcome measure 


(n=ll) 


(n=10) 


P value* 


Surgical performance: 


Total score (points) 


33 (25-39; 32-36) 


23 (21-28; 22-27) 


<0.001 


% reaching >30 points 


82 


0 




Operation time: 


Total time (minutes) 


12 (6-24; 10-14) 


24 (14-38; 20-29) 


<0.001 


Inter-rater agreement 0.79. y-coefficient 0.83 (95% confidence interval 0.68 to 0.98). 




*Mann-Whitney U test. 









The median total score on the general and task spe- 
cific rating scale reached 33 points (interquartile range 
32-36 points) in the simulator trained group and 23 
(22-27 points) in the control group (K0.001, table 2). 

The median total time to complete the procedure 
was 12 minutes (interquartile range 10-14 minutes) in 
the simulator trained group compared with 24 
(20-29 minutes) in the control group (P<0.00 1 , table 2) . 
Twenty one operations were assessed. 

The median number of simulated salpingectomies 
needed to reach the proficiency level in the simulator 
trained group was 28 (24-32 salpingectomies). The 
control group was offered simulator training after the 
study operation; nine of the 1 1 trainees in this group 
volunteered and a median of 26 (23-32) simulated 
operations were needed to reach the proficiency level 
(P=0.70). The mean time spent on training using the 
simulator was 7 hours and 15 minutes (5h 30 min-8h 
0 min) in the intervention group and 7 hours and 0 min 
(5h 15 min-7h 45 min) in the control group (P=0.65; 
table 3). The baseline score (first attempt) was 8 (5-15) 
in the simulator trained group and 9 (7-19) in the con- 
trol group after training (P=0.70; table 3). All trends of 
differences in baseline characteristics were not statisti- 
cally significant. 

The time used by the assessors to fill in the rating 
chart was the mean total operation time plus five min- 
utes for each DVD recording. The inter-rater agree- 
ment was 0.79 (166/210). The y coefficient used to 
investigate strength of correlations among the obser- 
vers at single subject level reached 0.83 (95% confi- 
dence interval 0.69 to 0.99). 

DISCUSSION 

Proficiency based virtual reality training in laparo- 
scopic salpingectomy compared with standard clinical 
education was associated with a clinically important 



Table 3 1 Number of sessions and duration of training in virtual reality simulator training 
programme in intervention group before training and in control group after surgery 

Simulator trained group 

Variable (n=ll) Control group (n=9)* Pvaluef 

No (range) of training sessions 28 (16-39) 6 (19-43) 0.76 

Duration (range) of training 7h 15m (4h 15m-9h 30m) 7h 0m (4h 0m-9h 15m) 0.70 

Median (range) score on first attempt (%) 8(5-15) 9(7-19) 

*Voluntary simulator training after surgery. 
fMann-Whitney U test. 
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improvement of operative skills during the actual pro- 
cedure. The learning curve in the operating theatre was 
also shorter. On the rating scale used in this study, 
which had previously been validated in a separate 
investigation, novices (fewer than five procedures) 
scored a median 24 points, and intermediately experi- 
enced trainees (20-50 procedures) a median 33 points 
compared with a median 39 points for experts. 19 The 
clinical implications of the present findings are thus 
extensive. After training in a specific procedure to a 
predefined (proficiency based) level inexperienced 
trainees progressed from the performance level of a 
novice to that of an intermediately experienced gynae- 
cologist, assessed in their first complex laparoscopic 
procedure. By using simulator training it might be pos- 
sible to bypass the early learning curve, which is 
known to be associated with an increased rate of 
complications. 20 This study was not designed to inves- 
tigate complication rates, and conclusions in this area 
must be drawn cautiously. In general it is difficult to use 
patient outcomes to evaluate a medical training course. 
Firstly, in contrast with trials of a single intervention 
(for example, a new drug) medical education is a com- 
plex intervention involving many interconnecting 
parts and different layers. 21 Secondly, assessment of 
surgical technical skills of individual trainees will 
need to be based on surrogate end points rather than 
outcomes such as morbidity or mortality because it is 
an ethical imperative that an operation performed by a 
supervised novice ought to have the same outcome as 
that of the supervisor. Training may cost time and 
some inconvenience for the patient but should never 
jeopardise safety or outcome. Thirdly, to show differ- 
ences in outcome, based on a training course, the num- 
bers of trainees should by far outnumber the total 
number of trainees available, thus making such a trial 
unfeasible. 

Operating time 

Although operating time might be greater with novice 
surgeons, the outcomes of a supervised operation 
ought to be the same. The time to complete the laparo- 
scopic salpingectomy was reduced by half. As the oper- 
ating theatre serves both productivity and educational 
purposes, shorter operation times are of benefit. 

The present results emphasise that by using virtual 
reality simulator training the surgical community can 
meet the need for proficiency based basic training in 
laparoscopy. These results also show that criterion 
based procedural training using a virtual reality simula- 
tor can help compensate for reduced working hours by 
bringing trainees to a higher level of performance more 
quickly. Traditional training depends on the supply of 
suitable procedures for training purposes, whereas 
simulator training can be used according to demand. 
To achieve an average of 28 salpingectomies can take 
a year or more in clinical practice, compared with eight 
hours of intensive training using the simulator. 

Finally, reducing the operating time by half, from 
24 minutes in the control group to 12 minutes in the 
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simulator trained group, might not seem important. 
Nevertheless it has been estimated that during resi- 
dency the additional costs of longer operating time is 
about (1997) $48 000 (£31841.00; €35 907.00) per 
graduate. 22 For more extensive surgery or just a bilat- 
eral procedure, increased use of simulator training 
could reduce novice operating time substantially, eas- 
ing the pressure on the often limited capacity for train- 
ing in the operating theatre. 

Transfer of skills 

Despite virtual reality simulators being introduced 
more than a decade ago, evidence on the effect of simu- 
lators on performance in the operating theatre is still 
sparse. 13 To date no published studies on the transfer of 
technical surgical skills from simulator to real opera- 
tions had exceeded grade 2a evidence; in our study 
the level of evidence is lb. These findings are sup- 
ported in a systematic review, 14 which also showed 
that the randomised controlled trials on effect of simu- 
lator training were generally of poor scientific quality. 
The major problems were randomisation with inade- 
quate concealment of allocation, lack of power calcula- 
tions or insufficient sample sizes, lack of observer 
blinding, insufficient training time or goals, non-stan- 
dardised comparators in control groups, and disparate 
interventions in intervention groups. Most studies also 
used surrogate end points such as bench station or 
simulator tests to establish the transferability of skills, 
instead of assessing skills in real operations. The con- 
clusion of the meta-analysis was that only few studies 
possess the necessary quality, that two studies showed a 
positive effect 2324 (real operation) and one study no 
effect 25 (simulated operation) of simulator based train- 
ing. Another common feature of the previous studies is 
that they were all carried out using basic skills rather 
than procedure specific simulation. We used a proce- 
dural simulator, which provides training in both psy- 
chomotor and cognitive skills, thereby also improving 
knowledge of the surgical procedure and its potential 
pitfalls. There are probably several reasons for the sig- 
nificant impact on both performance and time in our 
study compared with previously published studies. 
Firstly, the simulator provides a realistic graphic pre- 
sentation of anatomy in the surgical field and a good 
feedback system enabling the trainees to adjust their 
performance. Secondly, using predefined training 
goals (the expert proficiency level) instead of fixed 
time or numbers, encouraged the trainees to rehearse 
until they reached the maximal effect of the simulator 
training. Thirdly, instead of selecting medical students 
who might not choose a surgical career, we studied 
highly motivated trainees who needed to learn laparo- 
scopic skills for the future. 

We measured the impact of simulator training on a 
salpingectomy, which is an important gynaecological 
laparoscopic procedure and can be viewed as a key 
operation possessing all the core skills needed for most 
laparoscopic procedures. We did not test external valid- 
ity and reproducibility beyond the specialty of 
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gynaecology. However the skills that were improved 
can probably be considered as genuine core skills, 
which would be beneficial across the surgical specialties. 

The flow of an operation is based on both technical 
skills and knowledge of the procedure, and therefore 
we expect the impact of simulator training would apply 
to other laparoscopic procedures, although the effect 
would be more visible on the technical side than in 
the knowledge of procedural skills. These suppositions 
are supported by a contemporary Swedish study on 
procedural virtual reality simulator training of chole- 
cystectomy, which reached conclusions similar to 
those of the present study. 26 

Observer reliability 

In the present study the y coefficient showed that there 
was no systematic discrepancy among the raters. The 
inter-rater agreement also reached a sufficient level, 
which is evidence of a valid and reliable assessment. 

The investigation was carried out in the same way 
that a curriculum integrated training course most likely 
would be implemented. The internal consistency of the 
trial could have been higher if all the trainees had oper- 
ated in the same theatre, using the same technical 
equipment, and with the same supervisor and staff. 
However, by showing the effects of simulator training 
in settings closely resembling a regional simulator 
training course the external validity was improved. 
The primary investigator helped the trainee to use the 
simulator and introduced the different training mod- 
ules but did not teach laparoscopic techniques. The 
feedback on performance was based on assessment in 
the simulator. A designated supporter at the training 
session could, however, be a source of bias. A setting 
where trainees practise by themselves could eliminate 
this potential source of bias. Finally, performing 
laparoscopic surgery also consists of identifying dis- 
eased anatomy, communication, teamwork, decision 
making, 27 and leadership, alternative plans, and con- 
version to open surgery if needed. 2829 These non-tech- 
nical skills are taught in the currently existing virtual 
reality systems to a limited degree only. In this study 
we did not provide training in these non- technical skills 
or assess them. Simulator training should probably be 
considered only as a supplement or preoperative train- 
ing; further education and practice in the operating 
theatre as well as further development of more com- 
plex virtual surgical environments (hybrid 
simulation) 30 is still required. 

Conclusion 

It is possible to transfer skills acquired during profi- 
ciency based training using a virtual reality simulator 
to a real operation. Training in proficiency based skills 
should be incorporated in a comprehensive surgical 
training and assessment curriculum for all residents 
before they operate on real patients. This can poten- 
tially improve patients' safety and improve efficiency 
in the operating theatre. 
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WHAT IS ALREADY KNOWN ON THIS TOPIC 



The European WorkingTime Directive has put extra pressure on surgical training programmes 

Virtual reality simulators could contribute to the training of core skills for laparoscopy 

High grade evidence of the effect of virtual reality simulator training on real operations is 
sparse 

WHAT THIS STUDY ADDS 

Training using a virtual reality simulator improved performance in a laparoscopic procedure 
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